Condition-Based Production for Stochastically Deteriorating Systems: Optimal Policies and Learning

Production systems deteriorate stochastically due to usage and may eventually break down, resulting in high maintenance costs at scheduled maintenance moments. This deterioration behavior is affected by the system's production rate. While producing at a higher rate generates more revenue, the system may also deteriorate faster. Production should thus be controlled dynamically to trade-off deterioration and revenue accumulation in between maintenance moments. We study systems for which the relation between production and deterioration is known and the same for each system as well as systems for which this relation differs from system to system and needs to be learned on-the-fly. The decision problem is to find the optimal production policy given planned maintenance moments (operational) and the optimal interval length between such maintenance moments (tactical). For systems with a known production-deterioration relation, we cast the operational decision problem as a continuous-time Markov decision process and prove that the optimal policy has intuitive monotonic properties. We also present sufficient conditions for the optimality of bang-bang policies and we partially characterize the structure of the optimal interval length, thereby enabling efficient joint optimization of the operational and tactical decision problem. For systems that exhibit variability in their production-deterioration relations, we propose a Bayesian procedure to learn the unknown deterioration rate under any production policy. Our extensive numerical study indicates significant profit increases of our approaches compared to the state-of-the-art

Optimal data pooling for shared learning in maintenance operations

This paper addresses the benefits of pooling data for shared learning in maintenance operations. We consider a set of systems subject to Poisson degradation that are coupled through an a-priori unknown rate. Decision problems involving these systems are high-dimensional Markov decision processes (MDPs). We present a decomposition result that reduces such an MDP to two-dimensional MDPs, enabling structural analyses and computations. We leverage this decomposition to demonstrate that pooling data can lead to significant cost reductions compared to not pooling

Dedicated maintenance and repair shop control for spare parts networks

We study a repairable inventory system dedicated to a single component that is critical in operating a capital good. The system consists of a stock point containing spare components, and a dedicated repair shop responsible for repairing damaged components. Components are replaced using an age-replacement strategy, which sends components to the repair shop either preventively if it reaches the age-threshold, and correctively otherwise. Damaged components are replaced by new ones if there are spare components available, otherwise the capital good is inoperable. If there is free capacity in the repair shop, then the repair of the damaged component immediately starts, otherwise it is queued. The manager decides on the number of repairables in the system, the age-threshold, and the capacity of the repair shop. There is an inherent trade-off: A low (high) age-threshold reduces (increases) the probability of a corrective replacement but increases (decreases) the demand for repair capacity, and a high (low) number of repairables in the system leads to higher (lower) holding costs, but decreases (increases) the probability of downtime. We first show that the single capital good setting can be modelled as a closed queuing network with finite population, which we show is equivalent to a single queue with fixed capacity and state-dependent arrivals. For this queue, we derive closed-form expressions for the steady-state distribution. We subsequently use these results to approximate performance measures for the setting with multiple capital goods

Efficient Emission Reduction Through Dynamic Supply Mode Selection

We study the inbound supply mode and inventory management decision making for a company that sells an assortment of products. Stochastic demand for each product arrives periodically and unmet demand is backlogged. Each product has two distinct supply modes that may be different suppliers or different transport modes from the same supplier. These supply modes differ in terms of their carbon emissions, speed, and costs. The company needs to decide when to ship how much using which supply mode such that total holding, backlog, and procurement costs are minimized while the emissions associated with different supply modes across the assortment remains below a certain target level. Since the optimal policy for this inventory system is highly complex, we assume that shipment decisions for each product are governed by a dual-index policy. This policy dynamically prescribes shipment quantities with both supply modes based on the on-hand inventory, the backlog, and the products that are still in-transit. We formulate this decision problem as a mixed integer linear program that we solve through Dantzig-wolfe decomposition. We benchmark our decision model against two state-of-the-art approaches in a large test-bed based on real-life carbon emissions data. Relative to our decision model, the first benchmark lacks the flexibility to dynamically ship products with two supply modes while the second benchmark makes supply mode decisions for each product individually rather than holistically for the entire assortment. Our computational experiment shows that our decision model can outperform the first and second benchmark by up to 15 and 40 percent, respectively, for realistic targets for carbon emission reduction

Expediting in Two-Echelon Spare Parts Inventory Systems

We consider a two-echelon spare parts inventory system consisting of one central warehouse and multiple local warehouses. Each warehouse keeps multiple types of repairable parts to maintain several types of capital goods. The local warehouses face Poisson demand and are replenished by the central warehouse. We assume that unsatisfied demand is backordered at all warehouses. Furthermore, we assume deterministic lead times for the replenishments of the local warehouses. The repair shop at the central warehouse has two repair options for each repairable part: a regular repair option and an expedited repair option. Both repair options have stochastic lead times. Irrespective of the repair option, each repairable part uses a certain resource for its repair. Assuming a dual-index policy at the central warehouse and base stock control at the local warehouses, an exact and efficient evaluation procedure for a given control policy is formulated. To find an optimal control policy, we look at the minimization of total investment costs under constraints on both the aggregate mean number of backorders per capital good type and the aggregate mean fraction of repairs that are expedited per repair resource. For this non-linear non-convex integer programming problem, we develop a greedy heuristic and an algorithm based on decomposition and column generation. Both solution approaches perform very well with average optimality gaps of 1.56 and 0.23 percent, respectively, across a large test bed of industrial size. Based on a case study at Netherlands Railways, we show how managers can significantly reduce the investment in repairable spare parts when dynamic repair policies are leveraged to prioritize repair of parts whose inventory is critically low

Real-Time Integrated Learning and Decision Making for Cumulative Shock Degradation

Problem Definition: Unexpected failures of equipment can have severe consequences and costs. Such unexpected failures can be prevented by performing preventive replacement based on real-time degradation data. We study a component that degrades according to a compound Poisson process and fails when the degradation exceeds the failure threshold. An online sensor measures the degradation in real-time, but interventions are only possible during planned downtime. Academic / Practical Relevance: We characterizethe optimal replacement policy that integrates real-time learning from the online sensor. We demonstrate the effectiveness in practice with a case study on interventional X-ray machines. The data set of this case study is made available with this article. As such, it can serve as a benchmark data set for future studies on stochastically deteriorating systems. Methodology: The degradation parameters vary from one component to the next but cannot be observed directly; the component population is heterogeneous. These parameters must, therefore, be inferred by observing the real-time degradation signal. We model this situation as a partially observable Markov decision process (POMDP) so that decision making and learning are integrated. We collapse the information state space of this POMDP to three dimensions so that optimal policies can be analyzed and computed tractably. Results: The optimal policy is a state dependent control limit. The control limit increases with age but may decrease as a result of other information in the degradation signal. Numerical case study analyses reveal that integration of learning and decision making leads to cost reductions of 10.50% relative to approaches that do not learn from the real-time signal and 4.28% relative to approaches that separate learning and decision making. Managerial Implications: Real-time sensorinformation can reduce the cost of maintenance and unplanned downtime by a considerable amount. The integration of learning and decision making is tractably possible for industrial systems with our state space collapse. Finally, the benefit of our model increases with the amount of data available for initial model calibration while additional data is much less valuable for approaches that ignore population heterogeneity

Real-Time Integrated Learning and Decision Making for Cumulative Shock Degradation

Problem Definition: Unexpected failures of equipment can have severe consequences and costs. Such unexpected failures can be prevented by performing preventive replacement based on real-time degradation data. We study a component that degrades according to a compound Poisson process and fails when the degradation exceeds the failure threshold. An online sensor measures the degradation in real-time, but interventions are only possible during planned downtime. Academic / Practical Relevance: We characterizethe optimal replacement policy that integrates real-time learning from the online sensor. We demonstrate the effectiveness in practice with a case study on interventional X-ray machines. The data set of this case study is made available with this article. As such, it can serve as a benchmark data set for future studies on stochastically deteriorating systems. Methodology: The degradation parameters vary from one component to the next but cannot be observed directly; the component population is heterogeneous. These parameters must, therefore, be inferred by observing the real-time degradation signal. We model this situation as a partially observable Markov decision process (POMDP) so that decision making and learning are integrated. We collapse the information state space of this POMDP to three dimensions so that optimal policies can be analyzed and computed tractably. Results: The optimal policy is a state dependent control limit. The control limit increases with age but may decrease as a result of other information in the degradation signal. Numerical case study analyses reveal that integration of learning and decision making leads to cost reductions of 10.50% relative to approaches that do not learn from the real-time signal and 4.28% relative to approaches that separate learning and decision making. Managerial Implications: Real-time sensorinformation can reduce the cost of maintenance and unplanned downtime by a considerable amount. The integration of learning and decision making is tractably possible for industrial systems with our state space collapse. Finally, the benefit of our model increases with the amount of data available for initial model calibration while additional data is much less valuable for approaches that ignore population heterogeneity